Currently most image and video coding systems and standards such as MPEG-2 and JVT/H.264/MPEG AVC use transform based techniques followed by quantization and entropy coding for performing compression. Turning to FIG. 1, a typical transform based compression system is indicated generally by the reference numeral 100. An input to the transform based compression system 100 is connected in signal communication with an input of a transformer 110. An output of the transformer 110 is connected in signal communication with an input of a quantizer 120. An output of the quantizer 120 is connected in signal communication with an input of an encoder 130. An output of the encoder 130 is connected in signal communication with an input of a decoder 140. An output of the decoder 140 is connected in signal communication with an input of a dequantizer 150. An output of the dequantizer 150 is connected in signal communication with an input of an inverse transformer 160. An output of the inverse transformer 160 is an externally available output of the system 100. The key idea is that such transforms, such as the Discrete Cosine Transform (DCT), de-correlate the image signal and compact the energy of an image block into a few low pass coefficients, which after quantization and de-quantization could still represent the signal rather accurately. Nevertheless, this quantization/de-quantization process needs to be carefully designed in order to have the best possible subjective and objective quality. A great deal of past research was primarily directed to the 8×8 DCT used in JPEG and MPEG-like encoders, and has focused on the design of the quantization process, and in particular with regards to the optimal quantization step size that is to be used, coefficient distribution and dead-zoning mechanisms. One of the most important observations was that coefficient distribution, in most cases, followed a Laplacian distribution that enabled a more accurate modeling and design of the quantization process. This assumption is followed within the design of many modern codecs and encoders, including H.264 in an attempt to improve Rate Distortion (RD) performance.
However, although the Laplacian distribution holds true for many cases (including for some material coded with H.264), due to the introduction of a new smaller (4×4) transform in the H.264 standard, and the consideration of the standard for a wide range of applications including high definition TV, broadcasting, video-conferencing, and so forth, there are cases that such distribution does not always hold true. One such application in particular is the encoding of Film Grain content where distribution can be better approximated for certain coefficients using a Gaussian or generalized Gaussian distribution. This suggests that techniques used to better fit Laplacian distributions might not be appropriate for encoding such content, especially at high bitrates (small quantization step sizes) resulting in poor subjective and objective performance. Although some of the past research also suggests that the distribution could be different (Gaussian, Cauchy, Generalized Gaussian, and so forth), they generally provide rather complex models for the design of the quantization process.
It is well documented that the distribution of the DCT AC coefficients, follows in many cases a Laplacian distribution:
      p    ⁡          (      x      )        =            λ      2        ⁢          ⅇ                        -          λ                ⁢                            x                              
Based on this observation previous research tried to estimate a Laplacian distribution that could fit to the actual coefficient distribution of the source, and based on that design the weighted quantization matrices and the dead-zone that is to be used during quantization. The simplest way to find such a parameter is to use the standard deviation:
      σ    2    =                    ∫                  -          ∞                ∞            ⁢                                    (                          x              -                              E                ⁡                                  (                  x                  )                                                      )                    2                ⁢                  p          ⁡                      (            x            )                          ⁢                  ⅆ          x                      =                            2                      λ            2                          ⇒        λ            =                        2                σ            
This estimate is done separately for each one of the AC coefficients, while also the same process is independently applied to the chroma AC coefficients as well.
The above-described process enables an adaptive design of the quantization process, for example for the selection of the optimal reconstruction values or the design of the quantization matrices. Such a process could for example be performed where the quantization factor (Q) that also results in the smallest mean squared error is first determined:
                    MSE        =                ⁢                              min            Q                    ⁢                      2            ⁢                          (                                                                    ∑                                          k                      =                      1                                                              k                      =                      M                                                        ⁢                                                            ∫                                              kQ                        -                                                  Q                          2                                                                                            kQ                        +                                                  Q                          2                                                                                      ⁢                                                                                            (                                                      k                            -                            kQ                                                    )                                                2                                            ⁢                                              p                        ⁡                                                  (                          x                          )                                                                    ⁢                                              ⅆ                        x                                                                                            +                                                                                    ⁢                              ∫                          MQ              +                              Q                2                                      ∞                    ⁢                                                    (                                  x                  -                  MQ                                )                            2                        ⁢                          p              ⁡                              (                x                )                                      ⁢                          ⅆ              x                                      )                                =                ⁢                              min            Q                    ⁢                      2            ⁢                          (                                                                    ∑                                          k                      =                      1                                                              k                      =                      M                                                        ⁢                                                            ∫                                              kQ                        -                                                  Q                          2                                                                                            kQ                        +                                                  Q                          2                                                                                      ⁢                                                                  (                                                                              x                            2                                                    -                                                      2                            ⁢                            kQx                                                    +                                                                                    k                              2                                                        ⁢                                                          Q                              2                                                                                                      )                                            ⁢                                              λ                        2                                            ⁢                                              ⅇ                                                                              -                            λ                                                    ⁢                                                                                                          ⁢                          x                                                                    ⁢                                              ⅆ                        x                                                                                            +                                                                                    ⁢                              ∫                          MQ              +                              Q                2                                      ∞                    ⁢                                                    (                                  x                  -                  MQ                                )                            2                        ⁢                          λ              2                        ⁢                          ⅇ                                                -                  λ                                ⁢                                                                  ⁢                x                                      ⁢                          ⅆ              x                                      )                                =                ⁢                              min            Q                    ⁢                      (                                                            ∑                                      k                    =                    1                                                        k                    =                    M                                                  ⁢                                                      ⅇ                                                                  -                        λ                                            ⁢                                                                                          ⁢                      x                                                        ⁡                                      [                                                                  2                        ⁢                        kQx                                            +                                                                        2                          ⁢                          kQ                                                λ                                            -                                                                        k                          2                                                ⁢                                                  Q                          2                                                                    -                                              x                        2                                            -                                              x                                                  λ                          2                                                                    -                                              1                        λ                                                              ]                                                              ⁢                              ❘                                  kQ                  -                                      Q                    2                                                                    kQ                  +                                      Q                    2                                                              +                                                                      ⁢                                            ⅇ                                                -                  λ                                ⁢                                                                  ⁢                x                                      ⁡                          [                                                2                  ⁢                  kQx                                +                                                      2                    ⁢                    kQ                                    λ                                -                                                      k                    2                                    ⁢                                      Q                    2                                                  -                                  x                  2                                -                                  x                                      λ                    2                                                  -                                  1                  λ                                            ]                                ⁢                      ❘                          MQ              +                              Q                2                                      ∞                          )            
It has been claimed that the above could be easily evaluated to determine the best quantization values, and then determine a table of perceptually weighted coefficients experimentally. The decision of the base matrix is then performed by multiplying each MSE-optimal Q with the perceptual weighting, and then collecting them into a matrix, which is finally scaled and rounded as appropriate for the given bitrate.
Nevertheless, other work tries to disprove the claim of Laplacian distribution, with the claim that the distribution is closer to Gaussian or a Generalized Gaussian, i.e., following the distribution model:
            f      ⁡              (        x        )              =                                        v            ⁢                                                  ⁢                          α              ⁡                              (                v                )                                                          2            ⁢                          σΓ              ⁡                              (                                  1                  v                                )                                                    ⁢                  ⅇ                      -                                          [                                                      a                    ⁡                                          (                      v                      )                                                        ⁢                                                                                x                      σ                                                                                          ]                            v                                      ⁢                                  ⁢        with        ⁢                                  ⁢                  α          ⁡                      (            v            )                              =                                    Γ            ⁡                          (                              3                v                            )                                            Γ            ⁡                          (                              1                v                            )                                            ⁢        where Γ(·) denotes the gamma function
  (            Γ      ⁡              (        z        )              ≡                  ∫        0        1            ⁢                                    [                          ln              ⁡                              (                                  1                  t                                )                                      ]                                z            -            1                          ⁢                  ⅆ          t                      )and ν and σ are positive real values. Note that Γ(·) for n integer values becomes  Γ(n)=(n−1)! while for half integer arguments Γ(n/2) has another special form which is:
            Γ      ⁡              (                              1            2                    ⁢          n                )              =                                        (                          n              -              2                        )                    !!                ⁢                  π                            2                              (                          n              -              1                        )                    2                      ,where n!! is a double factorial. The first few values for n=1, 3, 5, . . . are therefore:
            Γ      ⁡              (                  1          2                )              =          π                  Γ      ⁡              (                  3          2                )              =                  1        2            ⁢              π                        Γ      ⁡              (                  5          2                )              =                  3        4            ⁢              π            
It is immediately observed that for ν=1 and ν=2, the above generalized Gaussian becomes a Laplacian or Gaussian PDF respectively. Maximum likelihood approach is used for fitting the statistics of a given image or sequence in order to provide the ν and σ parameters, which are then used in similar fashion as with the Laplacian case to determine the optimal quantization values.
The H.264 standard until recently did not provide any mechanism to benefit from such properties during the quantization process. More specifically, the standard does not specify how the encoder should perform rounding during quantization, while it only specifies how the decoder reconstruction levels are computed using uniform quantization. The H.264 reference software on the other hand made the assumption that coefficients, both DC and AC, satisfied a Laplacian distribution based on which fixed rounding (⅓ for intra and ⅙ for inter coding) was used. Unfortunately, the coefficients are frequently not Laplacian distributed, resulting in very poor subjective and objective performance within many H.264 encoder implementations, especially at higher bitrates and resolutions. This is partly because the standard only considered a 4×4 transform, thus de-correlating coefficients less efficiently, while also the equal step quantizers and the rather aggressive dead-zoning process tends to throw away most of the AC coefficients even at high bitrates. For this purpose, the 8×8 transform was reintroduced in the standard, while also quantization matrices were also adopted allowing a finer tuning of the quantization process. An alternative method was also proposed that allowed a modification of the reconstructed coefficient value with lower complexity than the quantization matrices process. Based on this approach, instead of performing the quantization and de-quantization of a coefficient equal to W using an equation of the form:
  Z  =                              int          ⁡                      (                                                                              W                                                  +                f                            Δ                        )                          ·                  sgn          ⁡                      (            W            )                              ⁢                          ⁢      and      ⁢                          ⁢              W        ′              =          Δ      ·      Z      where Z is the final quantized level, Δ is the quantization step-size, and f serves as a rounding term for the quantization process (see FIGS. 2A and 2B), quantization is performed using an equation of the form:
      Z    =                  int        ⁡                  (                                                                    W                                            +              Θ              +              f                        Δ                    )                    ·              sgn        ⁡                  (          W          )                      ,          ⁢            and      ⁢                          ⁢              W        ′              =                  (                              Δ            ·                                        Z                                              -          Θ                )            ·              sgn        ⁡                  (          Z          )                    where now Θ is an additional parameter that allows of an offsetting of the final reconstruction value (see FIG. 3). This effectively reduces the probability that a coefficient is set to zero thus resulting possibly in reduced subjective and even objective performance. Turning to FIG. 2A, the relation between an input signal W and an inverse quantized output signal W′ for a uniform quantizer with step-size Δ and f=Δ/2 is indicated generally by the reference numeral 200. Turning to FIG. 2B, the relation between an input signal Wand an inverse quantized output signal W′ for a uniform quantizer with step-size Δ and f=Δ/4 is indicated generally by the reference numeral 250. Turning to FIG. 3, an impact of Θ=Δ/4 within a quantization process is indicated generally by the reference numeral 300.